Papers with quality scores
Early-Exit and Instant Confidence Translation Quality Estimation (2026.eacl-long)
Copied to clipboard
| Challenge: | Quality estimation models are often opaque and computationally expensive, making them impractical to be part of large-scale pipelines. |
| Approach: | They propose an uncertainty-aware quality estimation model that matches previous approaches at a fraction of their costs. |
| Outcome: | The proposed method reduces evaluation costs by 50% and improves reranking performance. |
Kandinsky 3: Text-to-Image Synthesis for Multifunctional Generative Framework (2024.emnlp-demo)
Copied to clipboard
Arkhipkin Vladimir, Viacheslav Vasilev, Andrei Filatov, Igor Pavlov, Julia Agafonova, Nikolai Gerasimenko, Anna Averchenkova, Evelina Mironova, Bukashkin Anton, Konstantin Kulikov, Andrey Kuznetsov, Denis Dimitrov
| Challenge: | Text-to-image (T2I) diffusion models are popular for image manipulation, but also for video generation. |
| Approach: | They propose a novel T2I diffusion model based on latent diffusion that extends the base model for various applications. |
| Outcome: | The proposed model achieves high quality and photorealism and is 3 times faster than the base model. |
Chain-of-Interactions: Multi-step Iterative ICL Framework for Abstractive Task-Oriented Dialogue Summarization of Conversational AI Interactions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have introduced paradigm-shifting approaches in natural language processing, yet their transformative in-context learning (ICL) capabilities remain underutilized, especially in customer service dialogue summarization. |
| Approach: | They propose a single-instance, multi-step framework that orchestrates information extraction, self-correction, and evaluation through sequential interactive generation chains. |
| Outcome: | The proposed framework outperforms existing models and prompts in the customer service dialogue summarization domain. |
Is Human Scoring the Best Criteria for Summary Evaluation? (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on summary quality measure have shown that it correlates well with quality scores produced by human annotators. |
| Approach: | They propose to use a criterion that does not rely on human scores to judge summary quality . they propose to develop a method that can be used to determine the best measure from a family of measures . |
| Outcome: | The proposed measure could be used to determine the best summary quality measure from a family of measures. |
Neural Quality Estimation of Grammatical Error Correction (D18-1)
Copied to clipboard
| Challenge: | Grammatical error correction systems are expected to correct most learners’ writing errors, but in practice they often produce spurious corrections and fail to correct many errors, thereby misleading learners. |
| Approach: | They propose to use supervised learning to estimate the quality of GEC output sentences to help instructors decide whether to correct the errors or ignore them altogether. |
| Outcome: | The proposed model improves on a feature-based baseline and shows that the state-of-the-art system can be improved when quality scores are used as features for re-ranking the N-best candidates. |
UniSpec: Training-Free Speculative Decoding for Robust LLM Acceleration Across Languages and Hardware (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for speculative decoding ignore device-specific verification costs and lack of mechanisms to assess draft token quality. |
| Approach: | They propose a training-free, lossless speculative decoding framework that enables robust, plug-and-play LLM acceleration across diverse hardware configurations and languages. |
| Outcome: | The proposed framework outperforms existing training-free methods while maintaining identical output quality across different hardware environments. |
Are we Estimating or Guesstimating Translation Quality? (2020.acl-main)
Copied to clipboard
| Challenge: | A carefully engineered ensemble of pre-trained multilingual language models won the QE shared task at WMT19. |
| Approach: | They propose to use pre-trained multilingual language models to train quality estimation for machine translation. |
| Outcome: | A carefully engineered ensemble of pre-trained language models wins the QE shared task at WMT19. |
A Multi-persona Framework for Argument Quality Assessment (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for argument quality assessment do not consider multi-perspective evaluation due to subjective nature of arguments. |
| Approach: | They propose a multi-persona framework for argument quality assessment that simulates diverse evaluator perspectives through large language models. |
| Outcome: | The proposed framework outperforms baselines while providing comprehensive multi-perspective rationales on IBM-Rank-30k and IBM-ArgQ-5.3kArgs datasets. |
Centering-based Neural Coherence Modeling with Hierarchical Discourse Segments (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior studies of coherence focused on identifying semantic relations between adjacent sentences. |
| Approach: | They propose a coherence model which takes discourse structural information into account without relying on human annotations. |
| Outcome: | The proposed model performs state-of-the-art on automated essay scoring and assessing writing quality tasks. |
Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has found that explanations can easily convince users that inaccurate VLM predictions are correct. |
| Approach: | They propose to evaluate two complementary qualities of VLM-generated explanations via two quality scoring functions to improve their accuracy. |
| Outcome: | The proposed explanations improve accuracy on the A-OKVQA, VizWiz, and MMMU-Pro tasks by 11.1%, including a 15.4% reduction in falsely believing incorrect predictions. |
CritiQ: Mining Data Quality Criteria from Human Preferences (2025.acl-long)
Copied to clipboard
Honglin Guo, Kai Lv, Qipeng Guo, Tianyi Liang, Zhiheng Xi, Demin Song, Qiuyinzhe Zhang, Yu Sun, Kai Chen, Xipeng Qiu, Tao Gui
| Challenge: | Existing methods to train language models rely on manual design, perplexity, or careful prompt engineering. |
| Approach: | They propose a method that automatically mines criteria from human preferences for data quality with only 30 human-annotated pairs and performs efficient data selection. |
| Outcome: | The proposed method improves on human-annotated test sets and shows high accuracy on code, math, and logic domains. |
Automatic Evaluation for Text-to-image Generation: Task-decomposed Framework, Distilled Training, and Meta-evaluation Benchmark (2025.acl-long)
Copied to clipboard
| Challenge: | Existing MLLMs rely on commercial models such as GPT-4o for evaluations, but they are not universally accessible. |
| Approach: | They propose a task decomposition evaluation framework based on GPT-4o to automatically construct a specialized training dataset to break down the multifaceted evaluation process into simpler sub-tasks. |
| Outcome: | The proposed framework outperforms the current state-of-the-art GPT-4o evaluation framework with over 4.6% improvement in Spearman and Kendall correlations with human judgments. |
D2CS - Documents Graph Clustering using LLM supervision (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Document clustering does not inherently ensure thematic consistency. |
| Approach: | They propose a framework that constructs a similarity graph over document embeddings and applies iterative graph-based clustering algorithms to partition the corpus into initial clusters. |
| Outcome: | The proposed framework constructs a similarity graph over document embeddings and applies iterative graph-based clustering algorithms to partition the corpus into initial clusters. |
PROPER Agents: Proactivity Driven Personalized Agents for Advancing Knowledge Gap Navigation (2026.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to proactive assistance are anchored in what users express or can read, leading to unnecessary or mistimed interventions. |
| Approach: | They propose a framework that explicitly models user-specific knowledge gaps in a controlled manner. |
| Outcome: | The proposed framework improves on quality scores and win rates across multiple domains, achieving up to 84% gains in single-turn evaluation and consistent dominance in multiturn interactions. |